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Abstract 

The particle swarm optimization (PSO) algorithm, in which individuals collaborate with their interacted neighbors like bird 
flocking to search for the optima, has been successfully applied in a wide range of fields pertaining to searching and 
convergence. Here we employ the scale-free network to represent the inter-individual interactions in the population, named 
SF-PSO. In contrast to the traditional PSO with fully-connected topology or regular topology, the scale-free topology used in 
SF-PSO incorporates the diversity of individuals in searching and information dissemination ability, leading to a quite 
different optimization process. Systematic results with respect to several standard test functions demonstrate that SF-PSO 
gives rise to a better balance between the convergence speed and the optimum quality, accounting for its much better 
performance than that of the traditional PSO algorithms. We further explore the dynamical searching process 
microscopically, finding that the cooperation of hub nodes and non-hub nodes play a crucial role in optimizing the 
convergence process. Our work may have implications in computational intelligence and complex networks. 
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Introduction 

Biological entities and natural processes have always been the 
source of inspiration for the resolution of many real-world 
applications, one of which is to search the optimum of a problem 
under some certain conditions via the evolution of a population, 
namely the nature-inspired optimization algorithm. The advan- 
tages of self- adaption, self-organizing and self-learning in natural- 
inspired optimization approaches allows us to address many 
challenging problems pertaining to complex systems that cannot 
be solved by traditional methods. Hence much effort has been 
dedicated to the nature-inspired optimization algorithms in the last 
decades, as summarized in several reviews for succinctly recent 
advances on this topic [1-3]. 

Particle swarm optimization (PSO) is a typical nature-inspired 
optimization algorithm proposed by Kennedy and Eberhart [4], 
which is inspired by the social behavior of swarms such as bird 
flocking or fish schooling [5]. In nature, a bird usually adjusts its 
movement to find a better position in the flocking according to its 
own experience and the experience of birds nearby. In PSO, each 
artificial particle has its own position and updates its velocity 
according to the attractiveness of its best previous position and the 
best previous position of its direct interacted particles. Conse- 
quently, a group of interacted particles (the population size is 
commonly selected in the range 20-50 [1,6-9]) cooperate with 
each other to search for an optimum in the solution space. 

Since PSO was proposed, it continuously draws attention in the 
field of nature-inspired optimization. Shi and Eberhart [6] 
introduced the inertia weight coefficient to damp the velocity of 
particles iteratively and thus the scope of the search can be better 
controlled. Several years later, Clerc and Kennedy [7] proposed a 
strategy for the placement of constriction coefficients and derived a 



reasonable set of parameters. They proved that PSO with 
constrictions is algebraically equivalent to PSO with inertia weight 
coefficient from a theoretical way. The "constriction" version PSO 
has been the canonical particle swarm algorithm today. Moreover, 
Mendes and Kennedy [8] indicated that a particle is not only 
simply influenced by the best neighbor so that they introduced the 
fully informed PSO, in which particles are affected by all their 
neighbors. Kennedy et al. [9,10] found that PSO with large 
neighborhood would perform better on simple problems, while 
PSO with small neighborhood might perform better on complex 
problems. Peram et al. [1 1] took distance into account and 
proposed the fitness-distance-ratio based PSO that combats the 
problem of premature convergence, in which particles move 
towards nearby neighbors with better fitness instead of the global 
best solution. Due to its effectiveness and simplicity in implemen- 
tation, PSO has been widely applied in solving many practical 
optimization problems such as transportation [12], biomedicine 
[13,14], power systems [15,16], communication system [17,18], 
electronics and electromagnetics [19,20]. 

However, in previous literatures about PSO, the particles are 
completely [4,21] or regularly interacted [8,10]. Since the 
discovery of small-world phenomenon by Watts and Strogatz 
[22] and scale-free property by Barabasi and Albert [23] a decade 
ago, it has been realized that most real networks are neither fully 
connected networks nor homogeneous regular networks, but of 
small-world and scale-free topological characteristics. Much 
evidence has demonstrated that the structural properties play 
key roles in dynamical processes taking place on complex networks 
[24—26], e.g., in the evolutionary game dynamics, cooperators 
cannot survive on fully-connected networks, but some cooperators 
may have the chance to establish clusters to against the invasion of 
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Table 1. Optimization benchmark functions. 



Goal for 



Function name 


Formula 


z? 


Range [x mitv x max \ 


Optimum 


function 


Sphere 


/i(*0=£li(^) 2 


30 


[-100,100]° 


0 


0.01 


Rosenbrock 


/2fo)= Y£z\ (100(x^ +1 -{xfff + ixf- If) 


30 


[-30,30]° 


0 


100 


Rastrigin 


fs(xd= Eti ((^) 2 - 10 cos (2nxi d ) +10) 


30 


[-5.12,5.12]° 


0 


100 


Griewank 




30 


[-600,600]° 


0 


0.05 


Quartic 


/s(*/) = Eli ^(x/) 4 + m^m[0,l) 


30 


[-1.28,1.28]° 


0 


0.05 



All functions are implemented in 30 dimensions and are minimization problems. The optimum is the best solution for the function. The goal for function is used to 
evaluate the optimization is successful or not. The algorithm is run for 5,000 iterations. If the goal is not met by that time, it is considered that the goal will never be met 
and the optimization is unsuccessful. f\ and f 2 are unimodal functions; / 3 and / 4 are multimodal functions;/ 5 is the noise test function and is also unimodal function. 
doi:1 0.1 371/journal.pone.0097822.t001 
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(b) K=4 
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(d) K=49 



Figure 1. Networks with different values of /((the average degree). The network density grows from left to right. Here (a) is a ring-like 
network and (d) is the fully connected network. 
doi:1 0.1 371 /journal. pone.0097822.g001 



defectors and survive on a regular lattice. Strikingly, scale-free 
topology can lead to the emergence and domination of cooper- 
ation [27-29]. Such previous findings prompt us to wonder how 
scale-free topology that captures the interaction pattern among 
particles affects the PSO and if scale-free topology can offer better 
performance of the optimization process. To answer these 
interesting questions, we proposed a SF-PSO, in which scale-free 
network that incorporates the diversity of individuals is exploited 
to better mimic the real situation and improve the traditional 
PSO. We find that the SF-PSO balances the convergence speed 



and the solution quality, resulting in a much higher optimization 
performance than traditional PSO. We substantiate our findings 
and validate the SF-PSO by systematic analysis associated with a 
variety of benchmark functions used in the literature. 

Materials and Methods 

Canonical PSO Algorithm 

We adopt the framework of the canonical PSO and give a brief 
introduction. Suppose the size of the population is JV, where each 




(e) fl=3.25 (f) H=2 (g) tf=1.25 (h) H=l 

N 

Figure 2. Networks with different values of H (the network heterogeneity). H= J] K 2 /(N xK 2 ), where K is the average degree of the 

i=\ 

topology and K t is the degree of particle i. The network heterogeneity decreases from left to right, from up to bottom. 
doi:1 0.1 371 /journal. pone.0097822.g002 
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Figure 3. The comprehensive performance criterion S vs. the 
average degree /C (a). Thecomprehensive performancecriterionS vs. 
the heterogeneity H, (b). The size of the population N is 50 and each 
point is averaged over 100 times. Algorithm runs for 5,000 iterations 
each time. 

doi:10.1371/journal.pone.0097822.g003 

particle i has a position Xj = [x^,...Xi d ,...Xi D ]eRP and a velocity 
V/=[V/ 1 ,...v/,...v,- jD ]eR Z> in the Z)-dimensional solution space. 
Here pi is particle i's best previous position and p g is the best 
previous position of the best performer in i's neighborhood, pi and 
p g is updated iteratively. The particles are manipulated according 
to the following equation: 

vi d = xx [vf + ci x r i x (pi d - xf) + c 2 xr 2 x(p g d - Xi d )] ( 1 ) 




K 

(b) 

Figure 4. A typical BA scale-free network with: m 0 = 4, /??= 2 and 
N=50, (a). The cumulative degree distribution of the BA scale-free 
network, (b). 

doi:10.1371/journal.pone.0097822.g004 



x d = x d + v d (2) 

where C\ and c 2 are the acceleration coefficients, r\ and f2 are two 
uniformly distributed random numbers which are separately 
generated in the range [0,1]. X is tne constriction coefficient to 
control the convergence speed of the population [6] . 



Table 2. Performance of three PSOs. 
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The size of the population N is 50 and each value is averaged by 100 times. Algorithm runs for 5,000 iterations each time. The results of SF-PSO correspond to the 
average over 10 network configurations with m 0 = 4, m-2. 
doi:1 0.1 371/journal.pone.0097822.t002 
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Table 3. Convergence speed of three PSOs. 
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The size of the population N is 50 and each value is averaged by 100 times. 
Algorithm runs for 5,000 iterations each time. The results of SF-PSO correspond 
to the average over 10 network configurations with m 0 = 4, m-2. 
doi:1 0.1 371/journal.pone.0097822.t003 



The first component of Eq. (1) stands for the previous velocity 
that provides particle the momentum to search the solution space. 
The second component is the "self-cognitive" part that particle 
learns from the best previous position of itself. The third 
component, known as "social-cognitive" part, represents how 
particle optimizes the position by learning from the best previous 
position of its neighbors. Following common practices, 
a =c 2 = 2.05 and % = 0.7298 [1,8,30,31]. 

Benchmark Functions 

Benchmark functions are often used to evaluate the perfor- 
mance of optimization algorithms [8,10]. The formula, the 
number of dimensions, the admissible range of the variable, the 
optimum and the goal values are summarized in Table 1 . We use 
all these representative functions to test the optimization 
performance of our algorithm in comparison with traditional 
PSOs. 



different network densities (Figure 1). Secondly, we design a 
graphical scheme that enables a continuously transformation from 
a heterogeneous star-like network to a homogeneous ring-like 
network to examine the effect of network heterogeneity (Figure 2). 

All the topologies are tested in the framework of PSO for the 
optimization of five benchmark functions (Table 1). We use the 
comprehensive performance criterion S in Refs. [8,10] to evaluate 
the performance of different topologies. Since different functions 
are scaled differently, it is impossible to combine raw results from 
different functions. In this situation, we standardize the results of 
each function to a mean of 0.0 and standard deviation of 1.0. 
Thus, all results from different functions are normalized to the 
same scale. As all of these functions involve minimization, a 
negative result after standardization is better than the average. 
After standardizing each function separately, we can combine 
them and find the average as the value of S. 

As shown in Figure 3(a), neither the fully connected topology 
nor ring topology performs the best. The information dissemina- 
tion of fully connected topology is too fast to be able to escape 
from the local optima. The ring topology, on the contrary, 
prohibits the achievement of convergence, accounting for its bad 
performance. Topology with medium density can achieve the best 
result by making a balance of convergence speed and optimum 
quality (e.g., K— 12). From Figure 3(b), we can also find that a 
topology with a moderate heterogeneity performs better (e.g., 
H— 4.92). In high-heterogeneous topology, e.g. star-like network, a 
hub particle holds the dominate position and it can influence most 
of the population due to its large degree. Particles will soon 
converge to the hub, resulting in the tragedy of premature 
convergence. In homogeneous topology, e.g. ring network, 
however, no particle has a significant effect on the whole 
population. The convergence speed is too slow to induce good 
performance. Based on these preliminary results, we infer that 
network with moderate density and heterogeneity facilitates PSO. 



Comparative Study of Different Topologies 

In previously established PSOs, the population is assumed to be 
completely or regularly connected. However, the commonly 
observed structural characteristics in complex networks motivate 
us to explore the effects of various topologies on the performance 
of PSO. Firstly, we use the topologies from ring to fully connected 
to test the performance variation of regular topologies with 




F-PSO 
SF-PSO 
R-PSO 



1000 



Figure 5. Optimization process of three PSOs on Rastrigin 
function. The size of the population N is 50. The network parameters 
of SF-PSO are m 0 = 4, m = 2. Algorithm runs for 5,000 iterations. We have 
also examined other benchmark functions, and the results are alike. 
doi:10.1371/journal.pone.0097822.g005 



SF-PSO Model 

In 1999, Barabasi and Albert found the scale-free property in 
real networked systems and proposed the well-known Barabasi- 
Albert (BA) scale-free model whose generating mechanism can be 




Figure 6. The black square symbols stand for the average 
degree of particle i's neighbors K{ and the red circle symbols 
stand for the average degree of neighbors that particles learn 
from K" . Each data is averaged by 100 times on 10 network 
configurations with m 0 = 4, m = 2 and N = 50. Algorithm runs for 5,000 
iterations each time. 
doi:10.1371/journal.pone.0097822.g006 
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Figure 7. The variation of relative solution quality R K '(t). Each 
data is averaged by 1,000 times. The network parameters of SF-PSO are 
N = 50, m 0 = 4, m = 2. Algorithm runs for 5,000 iterations each time. 
doi:1 0.1 371 /journal. pone.0097822.g007 

described as "growth" and "preferential attachment" [23]: 
starting with m 0 connected nodes, at each step a new node is 
added with m (m<mo) edges that link to m different existing nodes 
and the probability P z that a new node connects to node i is 
related to i's degree Kf. 

Pi = Ki/J2j K J ( 3 ) 

where j runs over all the existing nodes. An important 
characteristic of scale-free networks is the degree distribution 
P(K), which is defined as the proportion of nodes in the network 



with degree if, and it can be described by a power law with a 
simple form: P(K)~K~ V . A variety of empirical researches have 
revealed that numerous real networks are actually scale-free 
networks [32,33] and scale-free properties have a significant 
impact on network dynamics [24-26]. 

A typical BA scale-free network is shown in Figure 4(a) and its 
cumulative degree distribution is presented in Figure 4(b). We can 
find that scale-free networks are neither dense as fully connected 
networks nor sparse as ring networks. Meanwhile, scale-free 
network is heterogeneous but not as heterogeneous as star-like 
network. In the following, we adopt the scale-free network to 
represent the individual interactions in PSO, namely SF-PSO, and 
further investigate its performance. 

Experiments and Discussions of SF-PSO 

Optimization Performance 

The SF-PSO is compared to PSO with fully-connected network 
(F-PSO) and PSO with ring network (R-PSO). The results are 
displayed in Table 2. We employ the main solution quality 
criterion R, which is the best solution at the end of the 
optimization, to examine the performance of algorithms. F-PSO 
obtains the best solution quality on f\, while SF-PSO ranks only 
second to it. SF-PSO obtains the best solutions on fj, fa and fs, 
which means that SF-PSO can achieve high quality solutions on 
unimodal function (fz), multimodal function (f-$) and noise function 
(fs). R-PSO achieves the best on while SF-PSO is just a little 
behind. Here we also examine the comprehensive performance 
criterion S. The result turns out that SF-PSO is remarkably better 
than other two PSOs, where SF-PSO is 25% smaller than F-PSO 
and 54% smaller than R-PSO. Obviously, SF-PSO's performance 
is outstanding according to solution quality criterion R and 
comprehensive performance criterion S. 
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Figure 8. The information dissemination ability of particles with different degrees, (a) I K {t) vs. iteration t. 7^(0= ^J2ie PK L i(t), where 
Li(t) is the times that particle i is learned by its neighbors at iteration t, p K is the collection of particles with degree /(and n K is the size of p K . (b) I K 
vs. particle's degree K. I K is the average performance of particles in the steady state, (c) Ik'(i) vs. iteration t. I K '(t) = I K (t)/K. (d) I K vs. particle's 
degree K. I K ' =I K /K. Each data is averaged by 1,000 times. The network parameters of SF-PSO are N = 50, m 0 = 4, m = 2. Algorithm runs for 5,000 
iterations each time. 
doi:10.1371/journal.pone.0097822.g008 
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Figure 9. Particles' contribution to population's activity, (a) G(t) 
vs. iteration t. G{t)= Ym= \ BM, where Bi(t) is particle i's ability of find 
a better solution at iteration t. The value of B t (f) is 1 when particle i's 
solution quality at iteration t Ri(t) is better than the previous best 
solution found by its neighbors p g (t— 1). On the contrary, the value of 
Bi(t) is 0. (b) M K vs. particle's degree K. M K = ± £f = , ^ iepK B t (t), 
wherep K is the collection of particles with degree /(and n K is the size of 
p K . (c) M K (t) vs. iteration f. M K (t)= ^E fePj , Ht). Each data is 
averaged by 1,000 times. The network parameters of SF-PSO are N = 50, 
m 0 = 4, m = 2. Algorithm runs for 5,000 iterations each time. 
doi:10.1371/journal.pone.0097822.g009 

According to the "no free lunch" theorem [34], any elevated 
performance over one aspect of problems is offset by performance 
over another. Since SF-PSO performs better in solution quality 
and comprehensive performance, there must be a cost in some 
other aspects. Next, we examine another important performance 
criterion of optimization algorithms, the convergence speed Qj 
which is the number of iterations required to accomplish the goal 
in Table 1. The results of three PSOs are shown in Table 3. It is 



found that F-PSO is of the fastest convergence speed while R-PSO 
is of the lowest. Due to the moderate network density and network 
heterogeneity, SF-PSO converges a little slower than F-PSO but 
faster than R-PSO. It is the "fate" of SF-PSO according to "no 
free lunch" theorem. 

Figure 5 shows the optimization process of three PSOs on 
Rastrigin in detail. We see that, F-PSO converges faster than the 
other two PSOs in the beginning. However, the population 
converges on a poor local optimum due to its "overspeed". For the 
R-PSO, the convergence speed at the early stage is the slowest, but 
the final solution quality is better than F-PSO. SF-PSO balances 
the convergence speed and solution quality, yielding the best 
performance in the comparison. 

Next we investigate the optimization microscopically to uncover 
the underlying mechanism in SF-PSO that facilitates the 
optimization process. 

Search Process of SF-PSO 

We suspect that the high-degree hubs and low-degree particles 
may play different roles in the optimization process. To confirm 
our speculation, we firstly explore what kind of neighbors that 
particles tend to learn from. We define K/ as the average degree of 
particle i's neighbors and define K/' = Y^2it = \ K/'if) as the 
average degree of neighbors that particle i learn from during the 
whole evolution, where T is the maximum iteration of the 
optimization and Kj'(f) is the degree of neighbor that particle i 
learn from at iteration t. Figure 6 shows that K/' is significantly 
larger than K/ , indicating that particles are more likely to learn 
from the neighbors with larger degree in their neighborhood, in 
other words, particles tend to learn from hubs. In the following, we 
use the scale-free network in Figure 4 to represent the interaction 
topology of particles. 

Since particles tend to learn from hubs, it is natural to examine 
whether the hubs have high solution quality or not. We define 
Rx(t) as the solution quality of particles with degree if at iteration 
t. Figure 7 represents the relative solution quality of particles with 
degree K: Rk \f) = Rx(f) / RK max (f) iteratively, and K max is the 
largest degree of population. We can find that at the early stage of 
the optimization, there is insignificant solution quality difference 
among particles with different degrees. As the optimization process 
evolves, the gap emerges. Since the largest degree of the 
population is 25, the relative solution quality of the particle with 
degree 25 is always 1 . The relative solution qualities of particles 
with medium degrees (^=8, 10) are between 1.0 and 1.1 at the 
end of optimization, while the solution qualities of particles with 
small degrees [K— 2, 3, 4) are above 1.1. It can be interpreted that 
hubs can obtain much more information and explore larger space 
due to its large degree. 

Further, we focus on the influence of particles with different 
degrees on information dissemination. The information dissemi- 
nation ability of particles with degree K at iteration t is defined as 
Ix(f)- Then, Ik denotes the performance of particles' information 
dissemination ability with degree K in steady state. Figure 8(a) 
shows the variation of information dissemination ability iteratively. 
It is found that, the advantage of hubs become increasingly 
significant during the evolution. Figure 8(b) shows that particles' 
performance on information dissemination in the steady state. 
Apparently, the performance of hubs is better than that of low- 
degree particles. Thus, it is natural to claim that hub particles play 
a crucial role on the information dissemination of PSO. To get rid 
of the influence of the degree factor, we further define two 
averaged indices: Tx(f) and Ik ■ As Figure 8(c) shows, in the 
beginning of the evolution, Ix(t) is around 0.25 for all particles. 
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However, with the increment of time step, Ik if) of medium- 
degree particles is still around 0.25 while high-degree particles and 
low-degree particles are obviously divergent. From Figure 8(d), we 
can also find that high-degree particles outperform low-degree 
particles in steady state. 

As we know, the optimization process of PSO (as well as many 
other nature-inspired optimization algorithms, such as GA [35], 
ACO [36] and SA [37]) is actually the process of finding "better 
solutions". If the particles find more "better solutions", the 
population is more active and facilitates the optimization of PSO. 
In this regard, we investigate the particles' ability on finding 
"better solutions", in other words, particles' contribution to 
maintaining the population's activity. 

Here, we define G(t) as the population's contribution to its 
activity at iteration t. As Figure 9(a) shows, G(t) firstly rises and 
then declines. At the beginning of the optimization, all particles 
are initialized with random positions and random velocities. As the 
optimization process evolves, particles will gather to the better 
regions in solution space according to the rule of PSO. If it is easier 
for particles to find better solution, G(t) will rises. When the 
optimization close to the optimum, it is hard to find better 
solutions and the value of G(t) declines. 

To distinguish the contribution of particles with different 
degrees, we define Mk as the average contribution of particles 
with degree K. Surprisingly, we find that the value of Mk 
monotonously decreases with the increment of degree, indicating 
that low-degree particles make more contribution to maintaining 
the activity of the population. The only chance for a particle to 
find a better solution is to defeat all neighbors according to the 
definition o^Pg in Eq. (1). Although a high-degree hub particle can 
collect more information through more links to obtain a 
considerable good solution, it is harder to find a "better solution" 
since it has too many rivals. As to a low-degree particle, it is just 
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